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Introduction 

With the advent of an all-volunteer recruitment policy, the United 
States Air Force is replacing automated assignment of enlisted recruits 
to career fields? with a guaranteed assignment program. Until recently, 
the majority of Air Force enlistees have been assigned to career fields 
by a computer assignment system. Early in basic training they were given 
an overview and audio-visual presentation of the fields available to them 
and asked to indicate their choices. The assignments, made by computer at 
the conclusion of basic training, used variables such as the number of 
persons needed in each field and the preferences expressed by the ei\listees. 

In 1971, USAF recruiters began offering, prior to enlistment, guaranteed 
assignment in any one of 26 Air Force career fields. Since then, the program 
has expanded so that now, in 1973, there are approximately 132 career fields 
from which an enlistee may choose. 

Should this program expand even further, the responsibilities traditionally 
assumed by recruiters would be altered considerably. In addition to promoting 
the Air Force as a career, recruiters "would be placed in the role of 
vocational counselors, guiding recruits to potentially far-reaching commitments. 
The implications for reenlistment are obvious. 

However, to guide men into rewarding careers, the recruiters will 
need either a much broader background in counseling than most of them now 
have or effective devices for measuring the vocational interests of recruits. 
Vocational interest inventories, or questionnaires, are used extensively for 
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this purpose by high school and college counselors. Unfortunately they 
are not appropriate for Air Force personnel. The questions, or items, do 
not reflect the nature of the activities of an enlisted man on duty. 
Furthermore, degree-of-job-satisf a .tion is not a factor in the interest 
scales, or scores, of the inventories used in schools and colleges. These 
scales reflect either the respondent's membership in an occupational group 
or his response to groups of homogeneous items. 

The Air Force needs a vocational-interest inventory for use in 
conjunction with their aptitudinal selection measures. The inventory 
should be designed specifically for Air Force recruiters who are administering 
a guaranteed assignment program. This report describes the initial stages 
of the development of such an instrument called the Vocational and Occupational 
Interest Choice Examination (VOICE). 
A Brief Review of Vocational Interest Measures 

Vocational interest measurement has one of the oldest and most 
successful records in the history of psychological testing. Two excellent 
summaries and evaluations of the published inventories can be found in 
Euros (1965) and Robinson, Athanasiou, and Head (1969). Much of this 
section is based upon Chapter 13 of Robinson et al, where concise evaluations 
of 13 interest inventories, also reviewed in Euros, are presented in addition 
to a review of one developed since Euros' publication. In this section, 
some of the difficulties and shortcomings of many interest inventories that 
render their use by the Air Force inappropriate are enumerated. The aim 
is not to unjustly criticize these inventories, because many of them, 
especially the Strong and Kuder, have been used with considerable success 
in guidance progvams but to explain why VOICE was developed for the 
Air Force's recruiting programs. 
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The Strong Vocational Interest Blank is one of the most highly 
regarded and well-researched inventories available. However, it has some 
psychometric problems, such as out-of-date norms and possible response set. 
Although the scale authors are continually making improvements, there is one 
problem that renders the Strong inapplicable for the Air Force; almost all 
the items and scales refer to occupations at the top fifth of the occupational 
status hierarchy. Very few refer to nonprofessional occupations, which 
largely make up the choices available to the enlisted man. Furthermore, 
the number of scales (112 on one form) makes interpreting a profile complex; 
and scoring the scales by hand is a tedious process, although a computerized 
scoring system could minimize this problem. 

The Kuder Preference Record — Occupational and Vocational are two 
highly rated and frequently used inventories. The occupational form was 
designed to compete with the Strong. The chief criticism of the Kuder 
forms has been against the interpretations of results based on the forced 
choice method of responding. Buros points out the difficulties of 
interpreting results in either a normative or an ipsative mode. It is 
believed that recruiters would have considerable difficulty making accurate 
and useful interpretations with either form. 

One recently published inventory which might have been considered 
for use by the Air Force is the Minnesota Vocational Interest Inventory. 
This inventory was designed to measure the nonprofessional occupational 
interests of men who enlist in the Navy. Development of the Minnesota 
Inventory, funded by the Office of Naval Research, began in 1946. In its 
evolution, the inventory moved from a military setting to a nonprofessional 
civilian setting. The item format is the same as that used in the Kuder 
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inventories and suffers from the same limitations created by the use of the 
forced choice format. Both occupational and homogeneous scales are presented. 
Buros does not review the Minnesota Inventory, but Robinson et al give it 
very favorable treatment. 

The Gordon Occupational Checklist was also designed for use in measuring 
nonprofessional vocational interests. It has been criticized because 
scores are given on a priori scales, with only an afterthought for item 
analysis. Also, there is some difficulty with the definitions of the 
scales, as well as with the underlying factor structure. 

Other inventories that could be considered are f:he Picture Interest 
Inventory, Guilford-Zimmerman Interest Inventory, How-Well-Do-You-Know- 
Your-Interests , Geist Picture Interest Inventory, Curtis Interest Scale, 
Fowler-Permeuter Interest Record, Career-Finder, and Qualifications Record. 
All of them have numerous shortcomings: most frequently, poor Item 
construction and analysis, potential response set, statistical problems, 
and psychometric inadequacies of reliability, horuDgeneity , discrimi^nation, 
and cross-validation. 
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SECTION II 
Development of Instruments 

An Overview 

Eight career fields were selected by the project directors, in 
consultation with the contract monitor, to serve as a basis for developing 
the interest inventory, ^ach field was in the guaranteed assignment 
program as of May 1, 1972, had a high inward flow of personnel, and was 
selected so that it represented two career fields from each apMtude index 
of the Airman Qualifying Examination — Mechanical, Administrative, Electric, 
and General. The Airman Qualifying Examination was the operational selection 
and classifying instrument used by the Air Force during the time period of 
the contract. 

After the career fields had been selected, the ETS project directors 
met with Air Force personnel in San Antonio and Washington, D. C, to 
discuss general considerations relevant to the project and a variety of 
more specific topics related to it. Discussion centered on the relation 
of the interest inventory to the needs of the Air Force for selection in 
both the guaranteed enlistment program and the automated assignment system. 
Background materials for writing items for the interest inventory were 
procured. Sampling considerations and the advantages and disadvantages 
of various plans were discussed. Job satisfactions were briefly discussed 
and a list of a priori percentages for job satisfaction was obtained for the 
eight selected career fields. VOICE became the acronym for the inventory. 

An item pool of 400 interest items was constructed at ETS and reviewed 
on the basis of job analysis data and job descriptions outlined in the 
Airman Classification Manual. This manual gives a detailed description of 
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each career field and the related fields that appear in the Dictionary 
of Occupational Titles (1969). A short job satisfaction questionnaire 
was also constructed. 

After approval of the inventory and questionnaire by Headquarters 
USAF, 4800 airmen were selected from bases throughout the world for 
field testing, validating, and norming the instruments. The inventories 
were mailed to the airmen through their Consolidated Base Personnel 
Officer (CBPO) . One follow-up mailing was sent to the CBPOs. 

To obtain information for comparative purposes, recruits in basic 
training were administered only the interest items of the inventory. 
This group — called men-in-general — was compared with satisfied airmen in 
each career field and also served in an attempt to estimate reliability 
coefficients. Multiple discriminant analysis and stepwise regression 
were used extensively in the development and validation of both the 
occupational and a priori scales. A cross-validation procedure was used 
to estimate the scale accuracy. 

Details of the Selection of Eight AFSC Career Fields 

To select eight career fields for the inventory, the project directors 
requested that background information provided by the Air Force include 
the Airman Qualifying Examination (AQE) entry requirements for a number 
of fields. The information received included a Uniform Aiirman Record 
summary of the frequency and percent of enlisted personnel assigned by 
AFSC as of December 31, 1971; an Airman Classification Chart; AQE 
requirements by career field; Air Force Specialty Codes included in the 
guaranteed assignment program; and three booklets about the guaranteed 
enlistment program. 

Using these materials, a list was made of all jobs in the Guaranteed 
^ A<3sfgninent Program for which Current Job Analyses were available. For 
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each job the frequency (N) of enlisted personnel assignment was indicated 
and also the minimum AQE aptitude requirement as of June 1971, Since 
it was desirable to select from each AQE aptitude index one career field 
with a high and one career field with a low score requirement, the career 
fields were sorted by type of aptitude requirement (i.e,, Administrative, 
General, Electrical, and Mechanical), then by high score requirement 
(at least 80 in Electrical, 60 in General and Administrative, and 50 in 
Mechanical), and then by low score (40 or less in Administrative, General, 
and Mechanical, and 50 or less in Electrical). This sorting resulted in 
eight groups of career fields. 

From each group, the three fields with the largest Ns were selected 
for consideration, except for General-high which had only two fields of 
potential interest. A total of 23 (7 x 3 = 21 + 2 = 23) career fields 
were considered for final selection. Brief descriptions of these fields 
were examined and discussed with the contract monitor, and on the basis 
of the magnitude of N, similarity, and judgment, the eight. fields given 
in Table 1 were selected. Career fields considered but not selected appear 
in Table 2. 

AQE score distributions for each aptitude type and job classification 
in each field were requested so the fields could be compared for overlap 
in AQE scores. No significant overlap was found for career fields in the 
same area. 

Developmenc of VOICE 

Construction of the interest scales began with an inventory of the 
ETS test collection (including 49 interest measures) and a review of all 
relevant occupational interest inventories. From these sources an item 
proof of 400 occupational interest items were written. It was decided, 

O 
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for scoring purposes, to assign related items to separate interest scales 
or groups, for example, mechanical, computational, clerical, and so forth. 
An a priori scale approach was planned for two reasons. First, the 
significance of response to single items is not very reliable. Second, 
the scale definitions would provide guidelines for writing items if 
alternate or parallel forms of the inventory were needed. Since occupational 
scales werd to be developed, a pool of miscellaneous items were also 
written. 

The i':;ems were based on two principal criteria: (1) each item would 
be assignee;, to one of 14 scales thought to be relevant to Air Force 
career fields (with the exception of the items assigned to the miscellaneous 
category) and (2) the language of the items should be appropriate for the 
probable reading level of recruited Air Force personnel. 

Each item presents an activity (e.g., "Tinker with a broken sewing 
machine") and asks the examinee to indicate whether he likes, dislikes, 
or is indifferent to the activity. The items were listed singly, not 
in groups. This iformat was chosen for its simplicity and efficiency. 
Zuckerman (1953) .',5 lowed that individual item arrangement is more efficient 
e:han criad arrange aent in terms of response time. Perry (1953) conducted 
another study in which the item pools were of equal size and triads were 
compared with individual items. Although a slight superiority was shown 
for the triads, it was not consistent. Items arranged individually also 
have the advantage of being easier to score. 

The scales originc^lly developed resulted from a thorough review of 
the major occupational int<*rest inventories extant and adaptations of them 
which were, in the judgment of the investigators, related to the Air Force 
Career Fields. In addition, a review of the jobs within the various career 
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fields suggested other scales which had no counterparts in other published 
inventories. In the course of writing the items,. it became apparent 
that some of the scales originally planned were not feasible because of 
the ambiguity of content or of the number of items that could be written* 
The result was a tentative set of 14 definable scales. These scales and 
the number of items in each are listed in Table 3. 

When the items were being written, two reference sources proved most 
helpful. One was AFM 39-1 (Department of the Air Force, 1970) which 
contained specific descriptions of Air Force jobs. The second was the 
Dictionary of Occupational Titles (DOT) (1969). It was thought that 
candidates for the Air Force would probably be more familiar with the 
nonmilitary counterparts of Air Force- jobs, so that, where possible, 
the DOT descriptions of jobs similar to those described in AFM 39-1 
were also used as source material. 

The next step was to send the 400 items in the proposed inventory 
format to the ETS Test Development Division for review. Revisions were 
made as a result of this review and the items were sent to the Air Force 
Personnel Research Laboratory for review and approval. Their helpful comments 
led to an extensive revision aimed at improving face validity and representing 
more accurately the job types available to enlisted men in the Air Force. 
In the course of the review process, approximately 20 percent of the original 
items were either modified or eliminated, and new items were substituted. 
In one case an entire scale (Textile Services) was eliminated. The 
final version of the interest inventory consisted of 400 items grouped 
in four general sections: Occupations (90 items). Work Tasks (210 items). 
Leisure Activities (70 items), and Desired Learning Experiences (30 items). 
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Table 3 

Original Scales for Item Pool 



Scale 


Number 
Of Items 


Scale 


Numb p r 
Of Items 


Mechanical 


37 


Computational 


3A 


Academic 


26 


Health Service 


36 


M-Scale 


34 


Pedagogy 


15 


Outdoors 


41 


Food Service 


20 


Scientific 


34 


Textile Service 


15 


Electronic 


34 


Audiographic 


15 


Clerical 


35 


Miscellaneous 


2A 
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The scales listed in Table 3 were revised, and the final version contained 
13 separate scales in addition to a pool of miscellaneous items. The 
scales and their item numbers are listed in Table 4, 
Development of Job Satisfaction Scales 

Prior to developing a job satisfaction scale, a search was conducted 
for existing job satisfaction scales and research dealing with them. 
The 13 scales given by Robinson et al (Chapter 5, 1969) were examined in 
considerable detail. It was concluded that there are several factors that 
may affect job satisfaction. In general, the factors could be characterized 
as either extrinsic or intrinsic. In order to include each significant 
factor, a draft scale of 47 items was developed. Four subscales were then 
devised, and the draft submitted to the ETS Test Development Division 
for review and to the Air Force Personnel Research Laboratory for review 
and trial testing. Comments and suggestions received from ETS's Test 
Development Division and from the Air Force were incorporated in the 
final version. 

Four job satisfaction scales were developed and titled, "Job," 
"Peer," "Supervision," and "Air Force^" The i6-item Job scale was designed 
to measure intrinsic satisfaction with the actual work activities. The 
10-item Peer scale was designed to measure satisfaction with one's 
fellow co-workers. The Supervision scale contained 12 items and measured 
satisfaction with one's supervisor* The Air Force scale was a brief 
8~item scale aiming to measure satisfaction with working conditions in 
the Air Force. Items were written in brief form, often consisting of 
only one word. Items were also stated in such a way that positive ^ 
response to some items would indicate satisfaction and to other items 
dissatisfaction. 
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The four job satisfaction scales are modifications of scales used 
in the Job Description Index (JDI) of Locke, Smith and Hulin (1965). 
An approach similar to that used in the JDI was considered appropriate 
because of its low verbal level and research indicating that the JDI 
scales have predictive, convergent, and discriminant validity, as well 
as internal consistency and stability (Robinson et al, 1969, pp. 105--107) . 
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SECTION III 
Field Test 

VOICE was administered in a field test to a sample consisting of 
4800 airmen, 600 from each of the eight selected career fields. To be 
in the sampling frame for a given career, an airman needed to have both 
AQE scores on file and at least six, but not more than 42, months of 
on-the-job experience. A simple random sample of personnel within each 
career field was selected from the airman tape files maintained by the 
Computational Sciences Division of the Air Force Human Resources Laboratory. 
Preparatory Steps 

The following information was obtained for each participant: his 
social security number (SSAN) , CBPO, and career field. The men were 
grouped -and listed by CBPO. The Personnel Research Laboratory supplied 
mailing labels for the selected airmen (2 sets) , mailing labels for the 
CBPOs (3 sets), rosters of selected airmen by CBPO (3 sets), initial and 
follow-up letters to CBPOs and airmen, and a magnetic computer tape 
containing the AQE scores. Air Force Service Career codes, and SSANs of 
the 4800 selected airmen. 
The Role of the CBPO 

The participants in the field test were distributed among 128 CBPOs 
throughout the world. Each participating CBPO was notified by headquarters 
USAF that he would be receiving an approved interest survey which should be 
distributed to identified personnel. Initial and follow-up mailings to 
the CBPOs, by the contractor, contained a letter from the Personnel 
Research Division outlining the project, instructions for survey administration, 
a roster of the participating airmen, and preaddressed sealed packets for 
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distribution to the men. Each sealed packet contained a VOICE booklet, 
answer sheet, introductory letter, pencil, and business reply envelope. 
The men were told to return their completed VOICE booklets directly to the 
ETS project directors. 

The initial mailing of VOICE to CBPOs began January 12, 1973. On 
January 19, a postal card was sent to each CBPO indicating that materials 
had been mailed and requesting cooperation by quickly distributing the 
materials. Follow-up mailings to nonrespondents began February 5 and were 
continued until March 9. 

In early March, the Contract Monitor telephoned CBPOs in the continental 
United States whose bases ETS reported had either a large nonresponse rate 
or a large nonresponse for the Security Specialist career. We were 
particularly interested in Security Specialists' responses because this 
group had a low response rate (less than 300 returns at the time), and it 
was believed that a minimum of 300 useable answer sheets from each career 
would be necessary to develop the scales. The response rate from those 
bases that were called increased significantly j particularly from those 
with large n'ombers of Security Specialist nonrespondents. 
Collection of the Con^leted Inventories 

To control the flow of the large amounts of material from the field 
test, a system of clerical updating was used for the inventories as they 
passed through the various stages of processing. An illustration of a 
control sheet appears in Exhibit 1. As can be seen, there was space 
for indicating such information as the participant's social security 
number (SSAN) , his name, general comments, answer sheet number (A/S#) , 
follow-up answer sheet number (A/S//2) , the date his material was received, 
the date of scan-editing, and ready for scoring (SCRIBE). 
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As the completed inventories were received, they were checked in, 
and the scan-editing process was begun. The purpose of the scan-editing was 
to check the answer sheets as follows: 

1. Checked to see that SSAN, AFSC, and A/S// were properly gridded; 

2. Identified inventories that were incomplete in any of the 
sections ; 

3. Identified inventories with "patterned" responses as well as 
multiple responses to items; and 

4. Identified respondents whose AFSC numbers did not correspond to 
those on record. 

Detailed scan-edit procedures are given in Exhibit 2. 

Inventories were collected until 3537 had been returned. Of those 
returned, 3104 were found to be useable. All AFSCs had more than 300 
useable returns, which was considered the minimum number required for 
analysis. Most returns found in the scan-edit to be unuseable were 
blank or incomplete, or the airman had been discharged or separated, or 
he reported himself to be in a different career field from that supplied 
by the Personnel Research Laboratory. A complete tally of the useable 
and unuseable returns by career field and reason for nonuse appear in 
Table 5. 

The response rates achieved for the various CBPOs were also 
calculated. They were generally high. Of the 128 CBPOs involved. 111 
achieved response rates of 60 percent or better. A list of the CBPOs 
from whom cooperation was requested is given in Table 6 with the number 
of inventories mailed to the CBPO and the response rate achieved. 
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Exhibit 2 
CONTROL AND SCAN- ED IT PROCEDURES 



Control Sheets are in order by answer sheet number. 



When answer sheet arrives 

1. Find proper control sheet. 

2. Fill in A/S// (for second mailing use A/S// (2)). 

3. Fill in date received. 

^. Fill in Scan-Edit date (see below for procedures). 

5. Check Ready for SCRIBE box if there are no scan-edit problems. 



Scan-Edit procedures are as follows: 

1. On back, check to see if A/S// is gridded; if not, grid it. 

2. Check to see that all items have been answered. Look also for 
obvious patterning and for multiple responses to one question. 
Note all discrepancies in general comment area of control sheet. 
Some of the possible problems are listed below. 

A. If Sections 1 to 4 are incomplete, put answer sheet in the 
appropriate stack. 

B. If Section 5 is incomplete, put answer sheet in appropriate 
stack. These will be scored but Job Satisfaction will not 
be rated. 

C. If obvious patterning occurs, put in appropriate stack. 
Decisions will be made at a later time. 

D. If there is a multiple response to one question, put in 
appropriate stack for a later decision. 

£. If AFSC does not agree with that provided by the AF, put 
in appropriate stack so we can check with the AF for 
resolution. 

F. If the airman has returned a blank answer sheet, put in 
the appropriate stack. These will be resolved later. 

3. Erase any excess marks on the paper. 

4. Fill in any partial marks. 

5. Turn to front page. Repeat Step 1. 

6. See if AFSC is gridded. If not, grid, using 0 in the fourth digit. 
See if AFSC is the same as that on the roster. 

7. Check that SSAN is gridded. If not, grid. 

8. Repeat Steps 2, 3, and 4. 

9. Tally AFSC numbers. 
10. Put in SCRIBE stack. 
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Table 5 
Test Administration 



Useable Unuseable, 
Returns Return£> 

AFSC Number Reason Number 







Incomplete 


1 *>i 




AHQ 


Patterned Responseis 


99 




JU J. 


wrong Rr ou rJumDer 








Completed by \7r0ng SSAN 


-1 

X 


4730 


3A6 


Answer Sheet Number Unknown 


5 


6711 


A67 


Returned Blank, no reason 


37 


7020 


385 


Claimed One Completed 


9 


8110 


315 


Separated or Discharged 


86 


Total 


310A 


Transferred or Reassigned 


25 






PCS 


33 






TDY 


11 






AWOL or Deserter Status 


4 






Patient Status 


2 






Military Confinement 


3 






On Leave 


1 






Addressee Unknown on Base 


22 






Total 


A33 



GRAND TOTAL: 3537 
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Administration to Basic Trainees 

In order to obtain a **men-in-general" group, as well as to estimate 
the test-re test reliability of VOICE, the inventory was administered to 
a group of airmen who reported for experimental testing during basic 
training. VOICE was administered to 312 men in their sixth day of 
training. A sample of 211 from this same group took VOICE again on their 
twenty-ninth and last day of basic training to attempt to estimate the 
reliability of VOICE. From this total, 209 valid cases were used as the 
men- in- general group. Some of the men from the original group were lost 
because of discharges, setbacks to other flights, sick call on the test 
date, and various other reasons. Since it was difficult to know in 
advance exactly how many airmen would report for testing, every man in 
each flight was tested until the 200 which the project directors had 
requested was reached. 
Data Transcription and Editing 

Useable answer sheets were batched (the unuseable ones were not 
processed) and the responses transcribed on magnetic tape by means of a 
special purpose scoring machine called SCRIBE (Scanning, Comparing, 
Recording Instrument for Better Education). A quality control check 
indicated that the probability of more than one error per 100.000 transcriptions 
was less than 0.01. All errors found were corrected. 

Data from the SCRIBE file were merged with the file obtained from the 
Personnel Research Division tape giving AQE scores for the participating 
airmen. This combined file was checked by the computer to see: 

1. Whether each airman on the SCRIBE file had corresponding AQE 

data. In each case, when records were found on the SCRIBE file 
without AQE scores, a check of the answer sheets for these cases 
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indicated that the SSANs had been grldded incorrectly. Corrections 
were made and the records matched, 

2. Whether each Side 1 of an airman's answer sheet had a corresponding 
Side 2 answer sheet. Errors of this sort resulted from incorrectly 
gridding the answer sheet number. Errors, when found, were 
resolved and the file corrected. 

3. Whether the AFSCs were valid in the sense of being one of the eight 
selected. Errors in these cases were also due to incorrect 
gridding. 

4. Whether each response was "in range." Responses to the interest 
items were coded by the integers 1 through 3, and items on the 
job satisfaction scale were coded 1 through 4. 

5. Whether five or fewer items not appearing in a block had been 
omitted. When' this occurred, the "indifferent" code was imputed 
for the interest items and the "sometimes" and "seldom" codes were 
imputed with equal probability for the job satisfaction items. 
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SECTION IV 

Statistical Analysis 

This section describes the statistical procedures that were used 
to identify satisfied personnel, develop both occupational and a priori 
interest scales, and cross-validate the procedures used in those scales. 
Scales were developed on the basis of airmen who exftressed satisfaction 
with their career field and on the basis of a group of airmen in basic 
training. First the scales were developed indepencl'^ntly in two half- 
samples and subsequently cross-validated. Final scales were determined 
by combining half-samples. 
Construction of Half-Samples 

Prior to analyzing job satisfaction data and developing scales, two 
half-samples w2re constructed. All airmen were grouped according to their 
career fields. Within each of the eight fields, each airman was ranked by 
his total AQE score. The total AQE score was the sum of the four normal 
deviates corresponding to the percentile scores usually reported for the 
scale. As a final step, each pair of airmen (in rank order) within each 
career field was considered and one of the two randomly assigned to 
Sample 1, the other to Sample 2, The result of this procedure was the 
formation of two half-samples of airmen in each of the eight career fields. 
The total AQE score was equal in expectation for half-samples in the same 
career field. The recruits forming the men- in-general group were also 
randomly divided. (This was a simple random division since no AQE 
scores were available for this group). Except where indicated, all of the 
analyses described were performed twice, once for each half^-sample. 
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Item Selection for A Priori Interest Scales 

To achieve more efficient scoring and eliminate heterogeneous items, 
a backward selection procedure wa3 used to reduce the number of items 
forming an a priori scale. All a priori scales were limited to a maximum 
of 16 items. In the case of scales with fewer than 16 items, the only items 
eliminated were those with negative correlations with the total scale score. 
The decision to use 16 items was based on the findings of Katz, Norris, 
and Halpern (1970) , who achieved internal consistencies above 190 for interest 
items in eleven of twelve interest scales, using a similar item response 
format. 

Identical procedures were used independently with each half-sample 
and, subsequently, the combined sample, to select items for the recommended 
a priori scales. The procedure is summarized below: 

1. Compute the correlation between each item in the pool assigned 
to a scale and the total score based on the remaining items 

in the same pool. Items were scored on the basis of 1 = dislike, 
2 = indifferent, 3 = like, and each item was given an equal 
weight in computing the total scale score. 

2. Select the item with the lowest item-total correlation and 
discard that item. 

3. Rescore the scale total, eliminating the item identified in the 
previous step. Recompute new item-total correlations. 

4. Repeat steps 2 and 3 until 16 items remain. 

5. Eliminate items with negative item-total correlations if fewer 
than 16 items were in the scale item pool. 

This process illustrates the logic of the scale construction. If 
this strategy were implemented on the computer, the several rescorings 
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would be time consuming and expensive. Identical results . obtained 
by manipulating the variance-covariance matrix of items. For each 
item p , the correlation with the total score based on the remaining 
items was computed by 



k k k - 

r^(t.p) - ( E S ,) / (( Z Z J (S . (1) 

P d=l i-1 d-1 

d^^p i^ip j^ip 

The quantity r^(t.p) indicates the correlation of interest. The 
notation (t.p) has been used in the subscript to show that the pth item 
has been removed from the total. The number of items is denoted by k , 
and the co variance of items i and j is denoted 

When the pth item was identified as having the lowest item'-total 
correlation, correlations were formed by calculating 



k k k , , 

r ,(t.pp') - ( I S ) / (( E E S,,) (S^,^,))*^^^ ... 

P d=l P J 1=1 d=l P P * 

d?^p i?*p d?*p 

d?«p' i?«p' d?«p' 

and the pth item with the lowest value of r^,(t.pp') was dropped. 
This process was continued until 16 items remained. 

The process just described was designed to achieve a high degree 
of homogeneity among scale items. Internal consistency coefficients 
(coefficient alpha) were computed for each scale after item selection 
as follows : 
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2 

where k represents the number of items in the scale, the item 

2 

variance, and the total scale variance for the k items. Whenever 

internal consistencies were calculated, as in the occupational keys and 
the job satisfaction scales, the same method was used. 
Identifying Satisfied and Dissatisfied Personnel 

A two-phase plan was initiated to assign individuals to each career 
field to satisfied and dissatisfied groups. First, within each field, 
satisfied and dissatisfied groups were formed according to their responses 
to the overall job satisfaction question (Item 417) . Personnel responding 
"very satisfied" or "moderately satisfied" were classed in the satisfied 
group, and those "moderately dissatisfied" or "very dissatisfied" were 
classed in the dissatisfied group. Second, a stepwise regression analysis 
(Draper and Smith, 1966, p. 171) was performed within each field to find 
the line&r function of the four satisfaction scores which best separated 
the satisfied from the dissatisfied personnel. This linear function was 
then used to rank all personnel within a career field in terms of "satisfaction." 
The first subjects with the highest scores were chosen as the satisfied 

group where corresponded to the number of subjects indicating overall 

satisfaction in Item 417. This resulted in the proportions of satisfied 
and dissatisfied personnel within each field being the same as those 
estimated from the overall question. 
Determining Occupational Scales 

Within each career field the zero order correlations between each 
interest item and a dichotomous criterion were computed. The dichotomous 
criterion was scored 1 if the subject was a member of the satisfied 
career field group and 0 for the comparison group, which was always 
men-in-general. The 50 items having the highest correlations (regardless of 
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slgn) with the group membership criterion were selected from the 400 items. 
These 50 items served as independent variables in a stepwise regression 
analysis with the dichotomous criterion. This procedure was equivalent to 
a stepwise discriminant analysis for the two-group case (Beaton, 1964) 
and served the purpose of selecting a final set of items • Items were added 
until the increment in the squared multiple correlation was less than 0.0025. 
The variables added up to this point constituted the occupational scale with 
unit weights, the weights arrived at through the stepwise regression analysis 
were retained for comparative purposes. It was necessary to reduce the 
search for items to comprise these scales from 400 to 50, because 400 
independent variables are too large for most stepwise regression computer 
programs. Also, scales consisting of as few items as possible make scoring 
easier. It was believed that 50 items would provide a manageable pool 
and be large enough to produce occupational scales of sufficient quality. 
Predicting Criterion Groups with A Priori Scales 

A similar procedure was used to arrive at the best linear function 
of a priori scale scores for separating each group of satisfied personnel 
from men-in-general within each career field. A stepwise regression 
analysis was performed, using the dichotomous criterion representing 
group membership. Scales adding at least 0.0025 to the squared multiple 
correlation were retained. 
Multiple Group Discriminant Analysis 

Using the 13 final a priori scale scores, a 16-group discriminant 
analysis (Cooley and Lohnes, 1962, Chapter 6) was performed on the eight 
satisfied and eight dissatisfied career groups. A similar 16-group 
analysis was also performed using the occupational scales. The purpose 
of the discriminant analysis was, in part, to validate the scales further 
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but also to check the positions of the various groups in the discriminant 
space. For example, the analysis would determine the proximity of the 
two groups from a similar career field in relation to groups from other 
career fields. Evidence of this nature would provide supplementary 
evidence as to the validity and usefulness of the scales developed. 
Cross-Validation 

The scoring weights derived from the series of stepwise riigression 
analyses in one half-sample were applied to the data in the other half- 
sample. Within each field the appropriate occupational scale score or 
linear function of a priori scales was used to predict whether the subject 
was classified as a member of the satisfied group or of the men-in-general 
group. Dissatisfied personnel were not included in the cross-validation. 
Classification rules which minimized the probability of misclassif ication 
were developed for each career. 

For each individual, the probability of membership in one of the 
two groups could be computed according to the formula (Cooley and Lohnes, 
1962, p. 138): 




where P[Hj|y_j^3 indicates the probability that the ith person with 
scores on the derived linear function of items or scales belongs 

to group . In this case, there are two groups and . The 

p^ is the proportion of satisfied personnel obtained in the cross-validation 
sample; the s^ is the estimate of the standard deviation of the linear 
function or score estimated from the analysis sample. 
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By taking the logarithm of the numerator of the quantity on the right 
side of the equation and substituting the sample quantities, the following 
classification rule is obtained: 



C = 



log 



N 

c 

NS 



2s' 



- m 1 m 

ijg -y- 

m 2s 



m 



(5) 



where is the number of subjects classified as satisfied in a given 

career group in Sample 2; is the number of subjects in the men-in-general 

group in Sample 2: N = N + N ; S and S are the standard deviations 
° I- > c m c m 

of Y estimated for the given career group and the men- in-general group 
respectively in Sample 1; and y^ are the estimated means of the 

function Y in Sample 1 for the career group and men- in-general group 
respectively. 

If C > 0 , a subject was classified in the career group. If 
C < 0 , the subject was classified in the men-in-general group. 
Hits and Misses were computed as follows: 

1. If C > 0 and the subject was in the career group, classify 
as a career hit. 

2. If C < 0 and the subject was in the career group, classify 
as a career miss. 

3. If C > 0 and the subject was in the men- in-general group, 
classify as a men-in-general miss. 

4. If C < 0 and the subject was in the men- in-general group, 
classify as a men- in-general hit. 

For the occupational scales the proportions of hits (correct 
classifications) obtained with unit weighting were compared with the 
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proportions of hits obtained using the exact weights estimated in the 
discriminant analysis. For the a priori scales, weights rounded off 
to integer values were compared to the exact weights. 
Obtaining Final Estimates 

Once the cross-validation was completed, data from the two half- 
samples were recombined and final scales were derived on the basis of 
all the data. That is, the steps performed for each half-sample were 
carried out for the entire sample. Satisfied and dissatisfied groups 
were defined, items for the a priori scales were selected, and occupational 
scales were developed using the entire sample of satisfied airmen within 
a career field as the criterion group and the entire men-in-general sample 
as the reference group, A final set of linear functions of the a priori 
scales were also developed in this way. The cross-validity estimates can 
be regarded, then, as lower bound estimates for the error in the classification 
rules based on the entire sample, since they are based on only half the 
data. 

In addition, all means, standard deviations, and intercorrelations 
were computed for the final scales, and test-retest correlations for all 
scales were computed on a subsample of 209 men- in-general. 
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SECTION V 
Results 

In this section the results of the statistical analyses are presented. 
Analyses relating to the job satisfaction questionnaire are presented 
initially, followed by those for the development and validation of the 
a priori and occupational scales. 
Characteristics of the Job Satisfaction Scales 

The four job satisfaction scales. Job, Peer, Supervision, and Air 
Force, appear to possess a sufficient degree of reliability and some 
discriminant validity, as can be seen in Table 7. This table gives the 
correlations between the four job satisfaction scales, as well as the 
correlation between the satisfaction scales and the single, overall 
job satisfaction item. The estimates of internal consistency have been 
placed in parentheses. 

The estimates of internal consistency for th« Job and Supervision 
scales were comparable to those reported for Smith's JDI (Robinson et al, 
1969, pp. 105-106), which were in excess of 0.80 for her five scales. 
Internal consistencies for the Peer and Air Force scales were coineiderably 
below that standard. Although the Air Force scale had only eight items 
and the lowest Internal consistency, the Supervision scale had the highest 
internal consistency, even though the Job scale had more items. 

Judging from the magnitude of the correlations between scales , the 
four scales do not appear to be statistically independent. This result 
was also reported for the JDI. However, the high correlation reported 
for the JDI between Work and People was not found between the VOICE scales 
for Job and Peer, even though the two pairs of scales are similar. These 
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Table 7 

Satisfaction Scale Reliabilities^ 
and Intercorrelations 



Scale 


Job Peer 


Supervision 


Air Force 


Overall 


Job 


(.840) .348 


.466 


.454 


.72.2 


Peer 


(.742) 


.476 


.351 


.260 


Supervision 




(.884) 


.379 


.343 


Air Force 






(.597) 


.373 



^Internal consistency coefficients are shown in parentheses. The 
correlations were based on the total sample of 3072 airmen. 
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positive correlations indicate that the VOICE job satisfaction scales 
may be u^easuring a general job satisfaction factor to a large extent. 

The four correlations between each of the scales and the single, 
overall job s^^tisf action item indicate that each scale is positively 
associated with overall satisfaction. However, satisfaction with the 
intrinsic nature of the work activities which a career entails, as 
reflected in the Job scale, is of major importance. 
Development of Rules for Classifying Satisfied Personnel 

Table 8 shows the multiple correlations for the total sample between 
scores on the job satisfaction scales and the single overall-satisfaction 
question. Within the eight careers, the correlations ranged from 0.68 
to 0.60. The standardized regression weights for the four scales indicate, 
with a high degree of consistency, that the score on the Job scale is of 
paramount importance in predicting how an airman responds to the overall 
job satisfaction question. When t-tests of the significance of standardized 
regression weights are performed, they show that in most instances the 
contributions of the remaining scales were not significant. Inspection 
of Table 8 also reveals that the regression systems for the eight careers 
were highly similar (in that weights for Job were high) , though the 
proportions of satisfied personnel differed within each field. Table 9 
shows the regression weights and multiple correlations for the half-samples. 

Total job satisfaction scores for each career field were generated 
using these regression systems. Groups of satisfied and dissatisfied 
personnel were distinguished by cutting scores determined by the proportions 
of satisfied personnel estimated within each career field. Table 9 
shows the regression weights and multiple correlations for the half- 
samples. 
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Total job satisfaction scores for each career field were generated 
using these regression systems. Groups of satisfied and dissatisfied 
personnel were distinguished by cutting; scores determined by the proportions 
of satisfied and dissatisfied responses to the overall satisfaction 
question. Table 10 shows the. numbers of satisfied and dissatisfied 
personnel in each career, along with the proportion of men in each field 
who were satisfied with their jobs. 

Since only satisfied personnel were to be used in developing scales, 
a desirable outcome would have been for substantial numbers of men within 
each career to have indicated satisr action with their work. Unfortunately, 
in five of the eight careers fewer than half the men indicated satisfaction 
with their jobs. Within Security Specialist only 24.4 percent were 
satisfied, which reduced the total sample of satisfied personnel within 
this field to 43 in Sample 1 and 32 in Sample 2. This suggests that an 
initial screening to identify satisfied personnel might have been useful. 
VOICE could have been administered to airmen identified as satisfied and 
to a sample of those who were dissatisfied. 
Selection of Items for the A Priori Scales 

The items for the VOICE a priori scales (Table 11) were selected 
independently for each half-sample, according to the statistical procedures 
described earlier. Since the objective was to construct homogeneous scales, 
the responses of every airman with a valid AFSC were used to choose the 
scale items regardless of the degree of job satisfaction the men expressed. 
The resulting sample sizes were 1537 for Sample 1 and 1535 for Sample 2. 

The items selected in the half-samples were relatively consistent. 
In six of the thirteen scales, identical items were chosen. Five scales 
had fewer than four items that were unique for half-samples. Only the 
Q cientific and Outdoor scales were substantially different. 

ERIC 



-39- 



4J 

0) O 
W) H 
oj ^ 

4J 

0) 0) 
O -H 

0) CO 
PLH -H 
4J 

(d 

CO 







r-l 




00 


o 


CO 




CO 


ON 


m 


r-l 


r-l 


CO 








00 


•St- 


m 




m 


m 


m 

























0> 








Q) 




•H 








0) 












b 




CO 


o 


00 








•H 


I— 1 


00 


O 








1— 1 




r-l 




CN 


(d 








c 




CO 








•H 


Q> 


0) 










1— 1 


•H 








4J 


9 


Q 


































CO 










CO 




13 








& 




0) 












•H 








o 








LO 


CN 


}i 




CO 


r-l 


r-l 




o 




•H 


rH 


r-l 






4J 








n3 




td 








0) 




CO 








•H 
























CO 
•H 












4J 












cd 




0) 








CO 




•H 








CO 




M-i 








•H 




CO 


ON 


CN 


CO 


Q 




•H 


r-l 


Oi 


o 






4J 


r-l 




r-l 




f-i 


cd 








1 


0) 


CO 
CO 










f-l 


•H 












Q 




















•H 














CO 










CO 












•H 




0) 








4J 




•H 








cd 






00 


r-l 


r-l 


CO 




CO 


o 


r-l 










r-l 


r-l 




o 












4J 




cd 












CO 








CO 
4J 












c 












0) 













CN 
ON 



ON 



00 
ON 



ON 



CN 
CN 



00 
00 



CN 
ON 



CO 
ON 



CN 
CO 



ON 



CN 
CO 



00 



o 

00 



m 
o 



o 



00 
ON 



o 

00 



CN 



CN 



00 



CO 



CO 



48 

















u 




















(0 
•H 




















iH 












cd 






4J 


(d 










(d 








CO 












g 




o 




•H 


u 


4J 
















r-l 


0) 


CO 








•H 




(d 




(d 


p- 


•H 






u 


Cd 


4J 


CJ 


(d 


•H 


CO 


r-l 








p- 


c 


0) 




a 




(d 






> 


QJ 




4J 




Q) 




•H 






u 




B 


a 






o 


a 






0) 








(d 


CO 




0} 






CO 




•H 


(d 






4J 


p< 






*n 


(d 




IS 




CU) 


(d 


CO 






o 


r-l 


C7* 






C 


M 










Q) 


W 






•H 


4J 


>> 






u 


piJ 






0) 


U 


CO 


u 




u 


0) 






(d 


r-l 


c 


•H 






0) 




O 


C 




a 




CJ 




r-l 


Q) 








a 




o 




:3 


(d 


U 


(d 




O 






a 


•g 


o 




Cd 


0) 


<d 


u 


•H 




a 




0) 


o 


a 


t2 




o 


< 


> 


< 


5 


CO 


H 



ERIC 



-40- 



so 





o 


m 


so 




O 


m 








SO 


rH 






















sO 






























o 




m 


>o 


iH 


a\ 




m 




rH 


SO 


O 


H 


iH 


iH 


iH 


CN 


CM 




m 


m 


in 


SO 




^ 


















CO 


cn 






















cn 






in 








O 


f5 


rH 








S 


















































s 


vo 






o 


M 


M 






























CM 


cn 


cn 




1— t 


o 


CN 


in 


r«. 


iH 


CN 


m 


St 


00 




















































r«. 


o 


CN 


if> 


r> 


iH 


•N 


St 


m 


St 


00 




oo 


CM 


CN 


CN 


CN 


m 


r") 


m 


St 


St 


St 


00 








































o 










5 




5 




3 


CN 




oo 




CN 




O 


























o 






so 


m 


CN 


m 


m 




CN 






















m 
n 






s 






m 


n 


m 


m 


m 


n 


cn 


cn 


cn 


rs 


\o 


r> 


as 


CN 




m 


>c 


00 


o 




























g 


CN 


«N 


CN 


CN 


CN 


m 


n 


m 


n 


n 


cn 




rs 


SO 


r> 




CN 




m 




00 


o 




o 






rs 




OO 




CN 


n 


n 


m 


m 


o 


IN 


<N 


CN 


CN 


CN 


n 


n 


n 


n 


cn 


cn 


St 


0^ 
























ON 


rH 




CN 




Pi 




in 








sO 




CN 


CN 


CN 


CN 


CN 


CN 


CN 


CN 


CO 


cn 


cn 














lO 


sO 




rH 




in 








NO 


vO 




r«. 


00 


00 


cn 




OS 


CN 


n 


n 


n 


n 


n 


n 


n 


n 


cn 


cn 




in 




n 


CN 


St 


m 


oo 




oo 


00 


in 




00 




iH 




St 




so 




r> 


r> 


00 








iH 




iH 


CN 


CN 


CN 


CN 


cn 


cn 


\£> 


m 




n 


CN 


St 


in 


00 




oo 


00 


in 


\o 


00 




iH 




St 


VO 


so 




r> 


r> 


00 








iH 


iH 


iH 


CN 


CM 


CN 


CN 


cn 


cn 


m 


o\ 




vC 




CT\ 




r«. 


CN 


OO 


o 


rH 


o 


ro 




in 








00 






in 


O 






iH 


iH 




iH 




rH 


CN 


CM 


CM 


cn 



so 
SO 



SO 



SO 
OS 



8 



CNl 



•S 2 , 

< it •Hi 



3 



O 

OO 



so 

•«* 


m 


•* 
rH 


so 
OS 
rH 


0\ 

o 

CN 


rH 

CN 


m 

rH 

CN 


St 

cn 

CN 


SO 

cn 

CN 


in 

so 
CM 


r«. 

rH 


0\ 
rH 


cn 


SO 
<T 


s 


s 


St 


00 
O 
rH 


s 

rH 


rH 


r«. 

rH 


rH 


r«. 

CM 


CM 

m 


so 
fO 




OO 

in 


S 


s 

rH 


O 

ON 
rH 


CN 


O 


rH 


CN 

OO 


cn 
oo 


\0 
00 


rH 
rH 

CN 


o 

CN 
CN 


OS 
CN 
CN 


m 
cn 

CN 


CN 


o 
IN. 


rH 
IN. 


CM 
00 


cn 
oo 




rH 
rH 

CN 


O 
CN 
CN 


OS 
CN 
CN 


m 
cn 

CN 


cn 

rH 


rH 

CN 


OO 


so 
CO 


rH 
CM 
rH 


CM 
rH 


^ 

rH 


St 

cn 

CN 


cn 
in 

CN 


SO 
CN 


rH 


rH 
CM 


OO 


so 
CO 


rH 
CM 
rH 


r«. 

CM 
rH 


rH 


PO 
CN 


cn 
in 

CN 


SO 
CN 



cn lA St 

00 O rH CO 



O 



00 
so 

cn 



St 


00 




rH 


CM 


CN 


CM 




00 


OS 


cn 


cn 


cn 


cn 


tn 


St 


09 




rH 


CM 


CM 


CM 




00 


OS 


cn 


cn 


cn 


cn 


r> 


cn 


00 


in 


00 


00 


o 


rH 


CN 


CN 


in 


CN 


CN 


CN 


CN 


cn 


cn 


00 


in 


oo 


00 


O 


rH 


CN 


CN 


in 


CM 


CM 


CM 


CN 


cn 




-41- 



The differences in the Scientific scales of the two samples can 
probably be attributed to their orientation. The eleven items unique 
to the Sample 1 scale can be associated with academic activities; for 
example,, reading about great scientists and studying astronomy, chemistry, 
meteorology, microscopes, nuclear reactions, physics, and radiation belts. 
These are all activities and courses likely to take place in schools and 
colleges. On the other hand, the items unique in Sample 2 represent a 
technical and operational orientation — performing experiments, determining 
concentrations, helping scientists, devising special equipment, determining 
the age of foods, keeping data records, classifying rocks, and using 
microscopes . 

As for the two Outdoor scales, the activities of Sample 1 are chiefly 
leisure activities, those of Sample 2 represent occupations. The Sample 1 
scale included canoeing, hunting, sailing, fishing, camping, playing softball, 
picnicing, and riding trail bikes. The Sample 2 occupations and activities 
are performed outdoors: longshoremen, lineman, lumberjack, mason, surveyor, 
mowing lawns, pouring concrete, planing trees, and roofing. 

Of course, these conclusions are speculative, but they do suggest 
considerations for further development of the a priori scales. The 
factor structure of the inventory and its relation to the a priori scales 
appear to be important. Since the a priori scales served as a model for 
constructing items, analysis of the factor stracture of the total inventory 
would serve as a test of the a priori structure and would possibly provide 
alternative formulations for broad area scales like the a priori scales. 

The items selected for the a priori scales, on the basis of Sample 1 
combined with Sample 2, are given in Table 12. In the Scientific scale, 
most of the competing items came from Sample 2; thus the scale is oriented 
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toward technical aspects of scientific endeavor. The Outdoor scale 
contains items predominantly indicating outdoor occupations, although 
there are a significant number related to outdoor leisure activity. 

As indicated previously, the a priori scales were constructed with 
the aim of selecting items that were homogeneous. Internal consistency 
coefficients were calculated, using coefficient alpha, for each scale 
in each half —sample and the combined sample . These coefficients are 
presented in Table 13. The Sample 2 coefficients were larger than those 
obtained for Sample 1 for each scale; the coefficients for the combined 
sample were between those obtained for Samples 1 and 2. The coefficients 
obtained were less than those obtained by Katz et al (1970, p. 38), who 
achieved internal consistencies above 0.90 for 10 of 12 scales, but they 
were approximately the same as those presented in the examiner manual for 
the Kuder Preference Record-Vocational (Kuder, 1956, p. 21). 
Selection of A Priori Scales for Cross-Validation 

In using a priori scales to discriminate the occupational interests 
of recruits from those of airmen satisfied with their specific careers, it 
is desirable to use fewer than the 13 scales. Moreover, it can be assumed 
that satisfaction in each career field depends upon various combinations 
of interests. For example, one would expect interest in mechanics to be 
essential to satisfaction with the work of a General Purpose Vehicle 
Repairman; one would not expect interest in mechanics to be a factor in 
satisfaction with work as an Administration Specialist. 

In order to determine which a priori scales to use for the cross- 
validation and what weights to apply to them, a stepwise regression analysis 
was performed for the scales developed for each half-sample. The dependent 
variable in these analyses was either satisfaction with a career field or 
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Table 13 



Internal Consistencies 
The Two Half Samples 


for the A Priori Scales 
and the Combined Sample 


for 






a 

Internal Consistency 




Scale 


Sample 1 


Sample 2 


Combined 


Aud iographic 


.8914 


.8983 


.8949 


Food Service 


.8907 


.8974 


.8941 


PedaeoKV 


.8906 


.9007 


.8957 


M-Scale 


.8131 


.8236 


.8167 


Leadership 


.8458 


.8515 


.8486 


Computational 


.9255 


.9317 


.9274 




.8985 


.9007 


.8996 


Scientific 


.9089 


.9293 


.9279 


Electronic 


.9404 


.9445 


.9425 


Outdoors 


.8230 


.8935 


.8711 


Mechanics 


.9208 


.9233 


.9227 


Clerical 


.8955 


.9006 


.8988 


Academic 


.8811 


.8932 


.8921 



Measures of internal consistency obtained by calculating coefficient alpha 




(6) 
2 

where k is the nuinber of items in the scale; s^ is the item variance; 
2 

and s^ is the total variance. 
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membership in the men- in-general group. Analyses were performed independently 
for each field and each half-sample. Scales were included until the increase 
in the squared multiple correlation was less than 0.0025. The resulting 
scales for each career field in each half-sample, the combined sample, 
and their weights appear in Tables 14 through 21. A negative weight in 
the tables indicates that individuals in the career field scored higher than 
individuals in the men- in-general group * Standardized regression weights, 
multiple correlations, and standard errors are also given in these tables. 

The scales indicated by the analysis to reflect satisfaction with a 
given career field are logical. For example. Mechanics was the first 
scale, and thus the scale with the highest zero order correlation with the 
criterion, selected among men in the two careers in the Mechanics AQE 
requirement group (Aircraft Maintenance S->ecialist and General Purpose 
Vehicle Repairman). Similarly, the Electronic scale was the first scale 
selected among personnel serving as Radio Relay Equipment Repairman and 
Ground Equipment: Repairman, the Computational scale among Accounting 
Specialists, and the Clerical scale among the Administration Specialists. 
Selection of Items for the Occupational Scale 

Occupational scales were developed for each of the eight careers. 
They were developed independently in each half-sample for cross-validation 
and in the combined sample for the recommended occupational scale. The 
stepwise regression technique used was equivalent to a discriminant 
analysis, which gives weights to independent variables (items in this case) 
so membership in one of two groups can be predicted with a minimum of 
error. The two groups considered were satisfied airmen in the career 
field for the scale being constructed and recruits serving as men- in-general. 
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The number of items selected for each scale, multiple correlation, 
and number of satisfied airmen in each career field are presented in 
Table 22 for each half-sample. Few items were selected for both Samples 1 
and 2 — no scale had mo!.e than ten. A detailed list of the items selected, 
along with the standardized regression weight, the regression weights, 
and their standard errors appear in Tables 23 through 31. Negative 
regression weights indicate an item was preferred by the men-in-general 
group. Thus, for the Weather Observer scale based on combined samples, 
satisfied members of the career field responded more favorably than 
men-in-general to studying meteorology, planing and taking care of a 
vegetable garden, studying calculus, making weather forecasts, visiting 
a museum, helping write questions for a test, writing a computer program, 
learning to navigate a boat, being a teacher, drawing blueprints for a 
bridge, and solving geometry problems. Compared to men-in-general, they 
tended to dislike marching in a parade, watching drag racing, organizing 
a military drill team, installing a telephone, fighting a fire, and 
constructing mathematical tables. 

Two sets of weights were retained for the half-sample analyses and 
cross-validation. One set was the actual regression weights; the others 
were plus or minus one, depending on the sign of the regression weight. 
Means and Standard Deviations 

Means and standard deviations for all final scales were computed 
for each career field and men-in-general and are presented in Table 32. 
All scale scores were first converted to a scale with a mean of 50 and 
a standard deviation of 10 for the entire sample of 3072 airmen in the 
eight career fields. The men-in-general data were not used in obtaining 
the conversion parameters, but means and standard deviations for the men- 
in-general are reported on the converted scale. 



Table 22 



Number of Items, Multiple Correlation, and Number of 
Satisfied Airmen in the Career Field 

SAMPLE 1 

Number 



Occupational Scale 


Items 


Multiple r 


Satisfied 


w Cd uitc? J. v/DocJlVcL^ 


19 


7S9Q 


108 


Radio Relay Repairman 


29 


. 791A 


111 


Ground Equipment Repairman 


28 


.7291 


71 


Aircraft Maintenance 


29 


.7090 


98 


Vehicle Repairman 


27 


.76A0 


92 


Accounting Specialist 


17 


.8289 


126 


Administration Specialist 


15 


.7108 


87 


Security Specialist 


38 


.7369 


A3 




SAMPLE 2 












Number 


Occupational Scale 


Items 


Multiple r 


Satisfied 


Weather Observer 


22 


.76A9 


117 


Radio Relay Repairman 


21 


.752A 


115 


Ground Equipment Repairman 


24 


.71A8 


72 


Aircraft Maintenance 


24 


.6385 


92 


Vehicle Repairman 


23 


.lllZ 


93 


Accounting Specialist 


18 


.8151 


132 


Administration Specialist 


23 


.6781 


9A 


Security Specialist 


27 


.7520 


32 
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Table 32 

Means (M) and Standard . evlatlons (o) for All Pinal Scales 



Weather Observer 



Audlographlc 

Food Service 

Pedagogy 

M-Scale 

Leadership 

Computational 

Health Service 

Scientific 

Electronic 

Mechanics 

Clerical 

Outdoors 

Academic 

Weather Observer 

Radio Relay Repairman 

Ground Equipment Rspalrman 

Aircraft Maintenance 

Vehicle Repairman 

Accounting Specialist 

Administration Specialist 

Security Specialist 

Job Satisfaction 

Peer Satisfaction 

Supervision 

Air Force 



Satisfied 
M o 

52.707 9.101 

51.821 10.904 

53.542 8.315 

49.588 9.378 

50.630 9.390 
52.819 8.925 
51.897 9.289 
55.302 8.105 
49.493 9.024 
49.335 9.630 
50.328 9.018 
50.422 8.762 
54.541 8.621 
57.935 7.271 
51.739 9.279 
48.961 9.347 
49.495 9.651 
45.267 8.052 
52.152 9.13? 

51.631 9.237 
49.787 9.301 
58.258 6.248 
55.905 7.177 
37.663 7.755 
55.372 7.913 



Dissatisfied 

M £ 

50.093 9.038 

51.391 10.127 

50.898 10.135 

45.603 9.396 

47.911 9.681 

48.687 9.833 

51.502 10.104 

52.691 9.203 

46.211 9.570 

46. 332 10.356 

45.905 8.569 
49.306 9.884 
54.4 70 9.580 
54.779 8.817 
52.651 9.005 
48.354 9.371 

48.906 9.915 
47.060 8.553 
52.807 8.886 
49.666 9.414 
51.011 9.817 
41.783 5.907 
50.019 9.479 
29.492 9.837 
46.746 9.798 



Radio Relay R*. ^airman 
S It ia fled Dl8. at isf led 



M 



M 



53,01? 9.568 50.609 .0.134 

50.506 10.150 49.400 9.448 

50.275 10.144 47.774 9.977 

49.420 9.128 46.109 7.856 

49.089 10.057 46.179 9.272 

52.199 9.225 49.229 9.695 

50.317 9.522 47.970 8.415 

54.106 8.958 51.252 9.086 

57.288 7.662 52.551 9.574 

52.951 8.746 49.976 9.879 

48.323 9.278 47.044 9.062 

51.595 9.328 49.719 9.927 

50.867 9.523 49.629 9.329 

52.021 9.108 51.120 9.475 

57.938 7.991 56.122 9.518 

52.673 9.421 52.272 9.716 

52.450 9.523 50.718 9.910 

50.980 8.543 50.947 8.022 

49.440 9.461 50.474 8.278 

48.391 8.879 .',8.278 8.813 

47.967 9.985 50.185 9.243 

58.550 5.203 43.025 6.223 

53.054 7.988 49.156 10.368 

36.924 8.126 30.007 9.705 

54.938 7.368 46.388 9.816 



Ground Equipment Repairman 

Satisfied Dissatisfied 
M o M 0 

51. 333 9.809 50.582 9.281 

51.172 10.681 49.775 9.787 

49.737 9.922 48.386 9.533 

52.972 9.555 50.516 9.820 

51.052 10.365 48.735 9.910 

49.597 9.812 49.846 9.594 

49.555 9.962 50.064 9.840 

49 . 304 10 . 052 50.352 10 . 0 30 

55.657 8.347 52.602 9.781 

54.720 8.798 50.210 9.176 

50.942 10.570 49.018 9.528 

52.880 9.662 50.219 9.617 

48.911 10.303 48.736 9.638 

47.748 9.631 49.165 9.603 

51.033 9.181 51.886 9.472 

56.646 8.055 53.180 8.566 

52.195 9.123 50.831 9.319 

54.478 9.276 51.239 8.511 

45.471 8.976 49.037 8.635 

48.433 9.242 50.432 9.719 

49.194 9.184 51.644 9.101 

57.878 4.898 42.529 6.623 

49.211 8.893 46.250 10.015 

32.889 9.122 26.072 9.185 



Aircraft Maintenance 



Satisfied 
M o 



Dissatisfied 



50,777 9.938 50.836 10.106 

50.436 9.446 49.191 9.645 

49.502 9.561 48.416 10.115 

53.837 9.336 51.168 9.858 
51.012 9.763 48.828 9.997 
47.911 9.102 47.506 9. 789 
49.510 9.699 49.505 9.973 
48.866 9.527 48.721 10.366 
52.599 8.690 51.100 9.698 
54.919 8.059 52.666 9.357 
49.668 9.673 48.675 9.831 
54.215 9.208 52,677 9.229 
47.477 9.046 47.695 9.237 
47.769 8.660 48.006 10.611 
49.033 9.635 49.898 9.278 
52.134 10.211 53.194 9.641 
54.806 8.908 53.549 9.847 
54.167 9.440 54.999 10.565 
45.410 8.136 47.305 8.707 
46.061 9.236 48.278 10.138 
48.632 9.720 51.394 11.076 
58.011 5.048 43.676 5.547 

51.838 8.455 46.859 9.152 
34.544 8.647 27.022 9.940 



53.381 8.792 46.330 8.666 52.611 8.736 44.533 9,335 
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Vehicle RepalnMn AccountlPg Speclallet Adalnletratlon Specialist Security Specl j allst Hen~In-General 

Satlafled Dleeetlef led Setlefled Dleeetlsfled Setlefled Dleaetlef led Satisfied Dissatisfied 

Mo MoMoMo li£M£lS£lS£ 

46.645 11.136 46.479 11.088 49.277 9.960 48.022 9.366 49.501 10.308 47.676 9.146 50.371 10.831 51.179 9.760 

47.527 9.781 47.424 9.732 50.718 10.402 49.763 9.208 52.638 10.587 47.812 9.307 50.138 9.864 49.528 9.388 

46.788 9.533 46.156 10.456 53.134 9.827 51.255 9.211 52.180 10.291 48.564 9.983 50.425 9.958 50.487 10.081 

51.995 9.848 50.452 11.213 48.632 9.552 46.372 9.213 51.730 10.241 49.171 9.911 55.943 10.410 52.241 9.784 

48.821 9.979 4C.729 9.980 53.676 9.121 50.867 8. 848 54.166 10.445 49.272 9.874 52.031 10.329 50.901 10.216 

45.416 9.295 44.358 9.416 57.774 8.562 45.920 8.125 51.938 9.566 47.679 10.056 49.393 10.348 48.497 9.660 

46.845 9.571 46.717 9.855 49.646 9.757 49.325 9.665 52.703 10.983 50.012 11.066 52.077 9.564 51.828 10.429 

45.499 10.318 45.960 10.383 50.878 9.462 49.806 9.348 48.840 10.177 46.731 9.698 47.930 10.611 49.232 10.155 

51,562 9.170 49.825 9.908 45.802 9.813 43.129 9.381 47.421 9.490 46.407 9.912 47.596 10.787 50.116 9.697 

56.120 7.495 53.023 9.055 46.301 10.287 45.285 9.523 47.758 9.609 45.562 9.839 48.565 10.541 49.607 9.834 

47.637 9.953 45.735 9.009 56.190 8.738 49.623 8.637 57.166 10.287 49.404 9.675 53.594 11.249 51.241 10.300 

51.664 10.431 49.924 11.701 46.360 9.120 46.944 9.524 47.306 9.474 46.877 10.558 49.686 10.222 51.826 9.972 

44.427 9.268 44.103 8. 652 52.166 10.042 52.774 9.566 52.021 10.355 47.962 9.948 49.126 9.741 50.377 9.918 

42.679 9.586 44.309 9.249 53.501 9.461 53.298 8.630 48.011 9.019 47.217 9.358 46.163 9.715 48.358 9.638 

47.254 8.895 48.480 10.043 47.776 9.310 49.906 8.915 43.730 8.773 45.942 9.434 41.521 9.004 48.740 10.492 

52.760 8.940 51.200 9.867 45.847 8.866 47.216 9.335 44.537 9.911 46.628 9.789 44.944 10.424 50.027 10.521 

53.109 9.085 51.593 9.928 46.933 9.304 47.615 9.473 45.875 9.506 45.519 9.769 45.630 10.512 50.337 10.539 

60.015 7.834 56.731 10.138 45.269 8.232 46.530 8.621 45.117 9.808 46.017 8.812 45.267 10.129 49.210 9.790 

43.455 8.630 45.348 8.238 60.826 8.226 57.375 8.994 49.686 9.113 48.355 9.859 45.493 9.6/6 47.648 8.950 

44.362 10.191 44.584 10.047 55.942 8.955 53.233 9*018 55.818 9.008 50.570 10.064 49.880 10.558 50.554 9.620 

47.084 10.250 47.586 9.804 47.918 9.374 50.546 10.123 51.711 13.329 50.272 10.424 57.045 8.882 52.101 10.001 

58.968 6.044 43.027 6.192 58.573 6.120 42.202 5.904 59.008 5.693 43.204 6.047 55.607 4.635 39.263 5.753 

51.410 9.009 44.715 10.155 53.720 9.025 48.497 10.575 52.638 10.923 46.758 10.658 48.440 8.529 47.440 10.872 

33.985 9.141 26.856 9.676 38.268 8.659 31.030 9.720 37.894 8.354 34.210 8.772 35.299 8.953 28.460 10.133 

51.163 10.157 43.757 8.979 54.775 8.256 45.495 9.395 55.959 9.308 49.634 9.706 52.944 8.863 45.928 9.947 



h 




50.264 


9.952 


49.033 


9.410 


50.477 


10.584 


55.302 


10.041 


51.978 


10.047 


48.082 


9.242 


51.053 


10.387 


48.824 


9.838 


49.546 


10.189 


49.848 


9.550 


50.939 


9.747 


50.099 


9.572 


49.294 


9.976 


42.958 


8.405 


40.171 


8.725 


41.124 


8.448 


41.463 


8.981 


42.790 


8.288 


40.473 


8.480 


41.911 


8.730 


42.012 


8.622 
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For the most part, the men-in-general means and standard deviations 
closely approximate the overall values of 50 and 10. The largest departure 
from the mean of 50 occurred for the M-Scale. This is not surprising 
since this particular scale entered early into the stepwise regression 
systems with more consistency than any other a priori scale. One might 
speculate that the difference is in large part the result of a shift 
in mean scores, which would occur as a recruit gains experience with military 
life. To put it another way, many of the activities or careers described 
in the M-Scale may lose some of their appeal once they are more directly 
encountered by an individual. Interestingly, satisfied personnel within 
all eight career fields had higher mean scores on both the M-Scale and 
the leadership scale than did their dissatisfied counterparts. Other 
a priori scales exhibited predictable patterns with high mean scores for 
satisfied personnel within the satisfied career fields logically related 
to a particular scale. Satisfied Accounting Specialists, for example, 
obtained a mean of 57.774 on the Computational scale versus a mean of 
45.920 for dissatisfied Accounting Specialists and 48.082 for men- in-general. 
Similarly, satisfied Radio Relay Repairmen obtained a mean score of 57.288 
on the Electronics scale versus 52.511 for those dissatisfied within the 
same career field and 49.546 for men-in-general. 

The occupational scale means exhibit a different but expected pattern. 
The highest means, of course, are obtained by the satisfied individuals 
in the career for which the scale was developed, the lowest for men- in-general, 
with the difference in means between these two groups averaging about one 
and one-half standard deviations. The dissatisfied personnel within the 
same career fields, however, achieved means almost as high as the satisfied 
personnel. A clear implication, on the basis of the occupational scales, 



"Ti- 
ts that almost as many dissatisfied personnel would bG placed in their 
present career fields as would satisfied airmen. 

One factor which may, in part, account for this result is the explicit 
selection which took place when the career personnel involved in the present 
study were originally assigned to their respective fields. In order to 
be assigned to the Weather Observer field, for example, a man had to have 
an AQE General score at or above the 80th percentile. Thus » the Weather 
Observer group was subjected to explicit selection on General AQE and to 
incidental selection on all variables correlated with General AQE. An 
item might reveal differences between the men-in-general group and the 
satisfied career groups merely because the item was subject to the effects 
of incidental selection. If most items in a scale fell into this category, 
one would expect the results to be fairly close to the results shown in 
Table 32. It can be seen that the a priori scales. which were not subjected 
to incidental selection during their construction exhibit a different 
pattern, with larger mean differences between satisfied and dissatisfied 
groups on the scales logically related to a given career group. Judging 
from the data presented in Table 32, use of the a priori scales on a purely 
logical basis would probably result in fewer dissatisfied personnel being 
reassigned to the same career. 

The method which employs a men-in-general versus a career criterion 
group has been used as the primary technique in keying occupational 
inventories. Clark (1961) sates "...a scoring key will be considered good 
if it does a good job of separating workers in a given occupation from 
workers in general." Campbell (1971) coiranents on the Strong Vocational 
Interest Blank: "The main purpose of men-in-general is to establish the 
general level of popularity of an item; this can then be compared with the 

ERLC 



rate of endorsement of the occupational sample to locate items that 

the members of the criterion sample answer differently from the reference 

sample." 

In the recommendations section, several possible alternatives to 
the traditional use of a men-in-general group are suggested. 
Correlations among the Scales 

Correlations among the various scales are given in Table 33. Those 
for the occupational scales, in the upper portion of the triangular matrix, 
are generally low, with the possible exception of a clustering of the 
career fields with Electrical and Mechanical AQE requirements. The interest 
correlations for these four careers tend to be moderately high, although 
considerably lower than those presented by Campbell (1971, pp. 36-41) for 
occupations of a similar nature. 

Correlations among the a priori scales are given in the right corner 
of the triangular matrix. Each correlation is positive, in contrast to 
that found by Clark (1961, p. 65) and Kuder (1956, p. 21) and similar to 
that obtained by Katz et al (1970, p. 33). The fact that some inventories 
report predominantly positive intercorrelations , while others report 
intercorrelations that are mixed in sign, most likely represents the 
differences in item format used. If a forced-choice format is used, as in 
Clark and Kuder, correlations of a mixed nature are like to result. On 
the other hand, if there is no constraint on responding to an item, a 
general willingness to respond either favorably or unfavorably will be 
indicated by the positive correlations. 

In order to get some notion of the structure of the relationship among 
the a priori scales, scales having intercorrelations greater than 0.60 
were identified. Similarly, scales correlating between 0.55 and 0.60 were 
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noted. The interrelationships of the scales were represented as a network, 
with the scales serving as points and the correlations above 0,60 as arcs 
forming the network (Figure 1). One can see three clusters forming. One 
represents Electronic-Mechanics-Outdoors-M-Scale activities, a second 
represents the remaining scales except for Food Service, which stands 
alone even though it relates somewhat to the scales in the second cluster. 

The purpose of presenting data in this form is to provide some 
suggestions for modifying the a priori scales in subsequent studies. 
It should be emphasized that the subject of this report is the initial 
phase in the development of an effective vocational interest inventory. 
The primary purpose of this phase was to develop an iteji pool, field test 
the pool, and construct experimental scales. Subsequent efforts should 
be aimed at modifying the experimental scales, adding new scales, and 
deleting redundant or misleading scales. Examination of Figure 1 may 
suggest modifications oi the a priori scales. For example, the Scientific 
and Pedagogy scales each have correlations of 0.60 or more with five 
other scales. Furthermore, four of the five high correlations are common 
for both scales. This suggests that it might be profitable to attempt 
to combine these scales. 

Correlations between the a priori scales and the occupational scales 
appear in the lower left corner of the matrix in Table 33. These 
correlations seem logical, in that scales for career fields that are 
mechanical in nature correlate highly with mechanical interest. The same 
holds generally for Electronic and Clerical career fields. 
Test-Retest Reliability Estimation for the Scales 

In order to estimate the reliability of the scales obtained from the 
combined sample, scores on the scales obtained from the two administrations 
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of VOICE to the subsaraple of 209 men-in-general were correlated. It should 
be recalled that these men were in basic training at the time of both 
administrations . 

The correlations were low in comparison to those for other interest 
scales. Correlations ranged from 0.35 to 0.55 for the occupational scales, 
with virtually all scales having correlations within 0.05 of 0.50. 
Correlations for the a priori scales were higher and ranged from 0.54 to 
0.74. Due to the fact that these correlations were considerably lower 
than those obtained by both Strong on the Vocational Interest Blank and 
Clark on the Minnesota Vocational Interest Inventory (they consistently 
obtained test-retest correlations of 0.80 or higher for their scales) 
and lower than one might expect by examining the internal consistency 
estimates, the validity of these correlations as reliability estimates 
must be questioned. 

In obtaining estimates of reliability for an instrument administered 
on two separate occasions, the instrument must be administered independently 
and under identical conditions. In theoretical terms, this means that the 
distributions must be identical for each trial; that is, the expected 
value or mean of the measurement must be the same for both administrations. 
Furthermore, the variances must be the same for both administrations. For 
detailed discussion of conditions necessary for a test-retest correlation, 
to serve as an estimate of reliability, the reader is referred to Lord 
and Novick (1968, Sections 3.3 and 3.4). 

To determine whether the correlations obtained were valid estimates 
of reliability, the conditions were examined to see how appropriate they 
were. The differences between the raw score means and raw score variances 
for each trial were obtained and are presented for each scale in Table 34. 

O 
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In addition, an F-test was performed to test the equality of the raw score 
variances, and a t-test was performed to assess the significance of the 
difference in means where the variances were concluded to be identical. 
The significance level used was the 0.05 level, with a "yes" entry in 
Table 34 indicating significance. 

Certain features of the data stand out. For example > all the 
differences in the means have t.he same sign for each type scale. Although 
this result is difficult to interpret for the occupational scale, since 
both positive and negative items weights comprise a scale, an interpretation 
can be made for a priori scales. Apparently, there was a greater tendency 
to dislike items on the second administration, which was reflected in the 
consistently positive mean differences. Also, with one exception, the 
raw score item variances for the first administration were larger than 
those obtained on the second administration. 

In examining the significance tests, six of 21 tests for equality 
of variance were significant. Eight of 15 tests for zero mean differences 
were significant. Only seven scales failed to achieve significant 
differences in either raw score mean or variance. These results, taken 
simultaneously, indicate that the conditions, which were necessary to 
validate the correlations obtained as accurate reliability estimates, 
were generally not satisfied. Thus, the obtained correlations cannot 
be used as reliability estimates. 

Further evidence of the lack of parallelism between the two VOICE 
administrations appears in Table 35, which gives the correlations among 
the a priori scales for the two administrations. The upper -right triangular 
matrix represents correlations obtained for the first administration, the 
lower-left triangular matrix represents correlations obtained for the 
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second administration. If one expected the conditions of the two 
administrations to produce equal scores and variances for both administrations, 
one would expect the correlations among the scales to be equal, at least to 
the extent achieved in the previous samples. This did not happen. The 
correlations for the second administration were consistently higher than 
those of the first, indicating more consistent responses to all items in 
the second administration. Since the correlations differed considerably, 
one must conclude that reliability was not being measured. 

The correlations obtained however may be of some use. If the means 
vary with time, as the data suggest, the correlations are known to under- 
estimate the true reliability (Cochran, 1970). 

The actual correlations obtained for each scale have not been presented 
since such a presentation might result in an inaccurate condemnation of 
the scale reliabilities. Estimation of scale reliabilities should be 
performed in the future under rigid conditions so that the estimates may 
be reported. 
Cross-Validation 

The percentages of correct classifications (hits) within each career 
field, using each of four different methods, are presented separately 
for each half-sample in Table 36. Two base rates are shown (at the far 
right) for comparative purposes. Base 1 is a "maximum blind strategy" 
in that it will yield the highest possible percentage of expected hits 
with the absence of any information that might relate individuals to 
groups. Under such conditions, classifying all individuals in the larger 
of the two groups will achieve the most hits. Base 2 represents proportional 
random assignment — a less than optimal strategy but one that might be 
employed in a situation where no useful information about individuals is 
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available, but where quotas must be met. Under this system, individuals 
are assigned at random to one of two groups until a quota is filled for 
one of the groups. The same classification proportions or probabilities 
were used to compute the base rates as were used in the classification 
functions. * 

Except for Security Specialist, all of the methods bettered the 
base rates with some consistency in both samples. Since so few individuals 
were classified as satisfied within Security Specialist, it is not 
surprising that the cross-validation yielded unimpressive results in this 
field. As Table 36 shows, the occupational scales appeared to be superior 
to the a priori scales for classifying individuals accurately. Nevertheless, 
Table 36 indicates that both types of scales possess considerable value 
for classification purposes. 

Though there was virtually no difference between the unit weights and 
exact regression weights for the occupational scales in Sample 1, there 
appeared to be a slight difference in favor of the regression weights in 
Sample 2 when the total percentages of correct hits were considered. There 
appeared to be little difference between the regression weights and the 
integer weights when the a priori scales were used to classify individuals. 

Table 37 shows the percent of correct classifications within career 
groups and men-in-general groups. Since the decision rules used for 
classification were aimed at minimizing the total number of errors, there 
tended to be some overassignment to the larger men-in-general group. It 
is important to recognize that the within-group hits do not imply that the 
interest scales developed are better for the men-in-general group. Had 
the satisfied career groups been larger than men- in-general, the overassignment 
would have been to the career groups. 
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Discriminant Analyses 

The first discriminant analysis with 16 groups and the a priori 
scale scores as Independent variables extracted 13 latent roots and 13 
corresponding discriminant functions. Table 38 presents the 13 vectors, 
or orthogonal linear functions, of the a priori scales. It may be seen 
that the first two functions accounted for most of the variation among 
the 16 groups. Table 39 presents the "centrolds" of each group on each 
discriminant function. 

Figure 2 presents the configuration of the 16 groups in the two 
dimensions, defined by the first two discriminant functions. The first 
discriminant function is characterized by a relatively large positive 
weight for the Computational scale and relatively large negative weights 
for the Electronic and Mechanics scales. The two Accounting Specialist 
groups had the highest negative positions on this discriminant function; 
the highest positive values were accorded the two Vehicle Repairman groups. 
The factor underlying the first dimension might be interpreted as a bipolar 
factor, with one end characterized by high interest in Computational tasks 
and low interest in Mechanics and Electronics and the other end characterized 
by high interest in Mechanics and Electronics and low interest in 
Computational tasks. 

The second discriminant function can also be interpreted as a bipolar 
factor. One end is characterized by high scores on Electronics and Science 
and low scores on Clerical, and the other end is characterized by the 
reverse pattern. The most extreme groups are the satisfied Radio Relay 
Repairman group and the satisfied Administration Specialists. 

In terras of the distances between groups on the first two discriminant 
functions, the satisfied and dissatisfied groups within a career field are 
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ia« Satisfied 
Q- Dissatisfied 

1 - Weather Observer 

2 • Radio Relay K r: -airman 

3 • Ground Equipment. Repairman 
U - Aircraft Maintenance 

5 - Vehicle Repairman 

6 - Accounting Specialist 

7 - Administration Specialist 

8 • Security Specialist 
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relatively close. This suggests that the interest patterns of the two 
groups within a field are L^ore similar than for groups in different 
fields, which is to be expected. Some exceptions do occur, however. 
For example, the satisfied Ground Equipment Repairman and dissatisfied 
Vehicle Repairman fall into a fairly tight cluster with the two Aircraft 
Maintenance groups. 

The second discriminant analysis was performed with the 16 groups 
again serving as dependent variables and the eight occupational scales 
serving as the independent variables. Table AO presents the eight latent 
vectors derived. Table 41 presents the centroids of each group on each 
discriminant function. The f.^.rst two latent vectors from this analysis 
can also be interpreted as bipolar factors with each having one high 
positive and one high negative weight corresponding to a scale. As 
Figure 3 shows, the configuration of the 16 groups on the first two 
discriminant functions is quite similar to that in Figure 2. 
Suggested Inventory Items 

The total number of items that would be needed to construct all 
scales in final form for an inventory is 246. These items are listed by 
number in Table 42. 
Conclusion 

Our statistical analyses indicate that the experimental inventory 
possesses considerable utility for distinguishing among career groups and 
for distinguishing between satisfied and dissatisfied personnel within 
career fields and a men-in-general group. The sample sizes used to develop 
the occupational scales were minimal at best and in some cases, inadequate. 
Campbell (1971) suggested using two independent samples of 200, stressing 
homogeneity and making sure that men who perform the occupation in some 
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■ - Satisfied 
O - Dissatisfied 



1 - Weather Observer 

2 - Radio Relay Ropav^naa 

3 - Ground Equipment riGpair.ria/i 
U - Aircraft Maiiitcn-irice 

$ - Vehicle Repairnym 

6 - Accounting Specialist 

7 - Administration Specialist 

8 - Secrrity Specialist 
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Table 42 





Suggested Items 


for 


Vocational Interest 


Inventory 




1 


62 


129 


205 


276 


335 


2 


63 


133 


206 


277 


336 


6 


66 


134 


207 


278 


337 


7 


67 


135 


209 


279 


338 


8 


68 


138 


211 


282 


340 


9 


70 


139 


214 


283 


341 


1" 


71 


141 


215 


285 


343 


11 


72 


142 


217 


286 


344 


12 


74 


144 


218 


291 


345 


13 


76 


145 


219 


299 


347 


16 


77 


149 


220 


300 


348 


17 


80 


150 


225 


301 


350 


18 


82 


151 


226 


302 


351 


19 


83 


155 


227 


303 


354 


20 


84 


156 


228 


304 


355 


21 


85 


157 


229 


305 


356 


22 


86 


160 


230 


307 


357 


25 


87 


163 


234 


308 


358 


26 


88 


166 


235 


309 


361 


27 


90 


167 


236 


310 


363 


30 


91 


168 


242 


311 


366 


32 


93 


169 


243 


312 


368 


35 


96 


170 


247 


313 . 


369 


36 


99 


171 


248 


314 


373 


39 


101 


174 


250 


315 


374 


41 


102 


175 


253 


316 


376 


43 


104 


176 


255 


317 


377 


44 


105 


177 


257 


319 


378 


45 


107 


182 


258 


320 


380 


46 


108 


183 


259 


321 


381 


47 


109 


186 


262 


322 


382 


48 


110 


187 


263 


323 


384 


49 


113 


189 


264 


324 


385 


52 


114 


190 


265 


325 


386 


54 


115 


192 


266 


327 


388 


55 


117 


195 


267 


328 


390 


56 


120 


196 


268 


329 


391 


57 


121 


199 


270 


331 


392 


58 


122 


201 


271 


332 


394 


60 


127 


202 


274 


333 


397 


61 


128 


203 


275 


334 


400 
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unusual form are not included. The sample sizes used tc develop the 
final occupational scales for the eight career fields of VOICE ranged 
from 75 for Security Specialist to 258 for Accounting Specialist. The 
cross-validation results which were based on only half the number used 
for the final scales reflect the differences in sample sizes in the 
different careers. In fields where large samples were available, the 
occupational scales generally crossed better than in fields where smaller 
sample sizes were available. The cross-validity estimtes are, of course, 
lower bound estimates of validity since they are based on only half the 
sample. 
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SECTION VI 
Re c ommend a t 1 o na 

For most of the Air Force career fields included in the field test of 
VOICE, the validity of the occupational and a priori scales as measures 
of recruits' vocational interests has been demonstrated. The evidence 
presented in th:«s report indicates that by means of these scales it is 
possible to classify men with considerable accuracy in either an 
occupational or basic recruit group. 

However, a number of problems have come to light which require 
solution prior to the operational use of VOICE. Among :he8e problems 
are the similarity of the Interest patterns observed In dissatisfied 
and satisfied groups within a career, the rather small criterion groups 
which were available, and the apparent shift in the interest patterns 
of basic recruits over a period of three weeks. Although a final set 
of items is recommended for the inventory, it is suggested that the 
following additional research be undertaken. 

1. Conduct a Longitudinal Study of Interest 

In this development effort, the scales of the inventory 
were validated using a standard cross-validation technique. 
Although this method of validating measures is worthwhile, 
it fails to examine the facet of validity that deals with 
prediction. In other, words, do the interest scales, or scores, 
obtained early in an airman's service predict his subsequent 
satisfaction with his Air Force career field? This is an 
essential aspect of validity that must be confirmed prior 
to the adoption of any measure of vocational interest. 
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Associated with this problem is the question of whether 
interests change as a result of experience in a career. It 
should be recalled that the mean scores, produced in this field 
test for the occupational scales, showed that airmen dissatisfied 
with their careers were closer in their interests to satisfied 
airmen in the same career than they were to men-in-general. One 
reason for this may be that airmen who are dissatisfied with a career 
field do, nevertheless, acquire some interests in common with those 
of men who like the field. Knowledge of the job, alone, may cause 
this. Tht validity of this hypothesis needs to be examined. 

Also, a loiigitudinal study would allow the examination of the 
stability of many items comprising the M-scale. For example, 
two items, "March in a parade" and "Take part in a military drill," 
appear in a nujnber of scales. These items appeared to identify 
satisfied personnel, in that men satisfied with their careers 
tended to dislike these activities more than men- in-general. 
However, this may have been due to the fact that the men- in-general 
had not been extensively exposed to such activities. After some 
experience in military service, recruits* interest in these 
activities may decline regardless of their satisfaction with 
their careers. By identifying items such as these, which may 
correlate with time and thus be useless for prediction, the 
validity of the scales would be increased. 

Such a longitudinal study could be conducted by following the 
men-in-general group used in this field test. Repeated annual 
administration of VOICE to these men would provide an initial 
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group, to be followed by subsequent groups of new recruits 
until a sample size is reached that would allow significant 
longitudinal inference, 
Ey"'»a nd the Men-In-General Group 

Closely associated with the longitudinal study could be the 
establishment of accurate statistical data for men-in-general. 
The importance of obtaining sound estimates for the distributions 
of responses cannot be overstated. The men- in-general group is 
used in either the development or validation of all scales for 
VOICE. 

Strong (1954, Chapter 21) illustrates the effects of different 
men- in-general groups on correlations, scoring weights, and 
occupational clusters. His first men- in- general sample consisted 
of a few thousand men he happened to test during development of 
the Vocational Interest Blank. He modified this sample by using 
various U. S. Bureau of the Census statistics on employed men. 
Since then, several revisions of the men- in-general group have 
taken place. The group developed in 1969 includes some 1,000 men, 
stratified by general occupational area and specific occupation 
(Campbell, 1971, pp. 398-399, for a detailed listing of occupations). 

The statistical standards of the men- in-general group used to 
test VOICE do not compare to Strong. We urge that the group be 
revised until it possesses sufficient quality. Under simple 
random sampling, a sample size of 625 airmen ensures that the 
standard deviation of a proportion will be less than 0.02. 
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The precision could undoubtedly be increased by using 
stratified sampling. Men- in-general could be stratified 
by numerous variables (AQE score range, career field, and 
biographical information are possibilities) in conjunction 
with other studies of their interest. There is no doubt that 
scales developed with such a sample would be more accurate. 
Test the Factor Structure of the A Priori Scales 
As stated in the section describing the development of the 
a priori scales, an underlying structure was hypothesized for 
the inventory, and items were written accordingly. Whether 
that structure was, in actuality, realized is open to question. 
A factor analysis of the items in VOICE would serve to answer 
the question. The analysis could be conducted immediately 
without collecting additional data. 

The results of a factor study might indicate that the a priori 
scales should be modified. For example, the factor analysis 
might suggest that the Pedagogy and Scientific scales be combined 
or that the Outdoor scale be divided into two separate scales. 
Revise the Job Satisfaction Scale 

The evidence gathered indicates that the nature of the work 
itself is of overwhelming importance in determining how satisfied 
an airman is with his career. Rather than include scales which 
attempt to measure other factors, it is suggested that the job 
content scale be made more reliable by adding items and performing 
an item selection such as that performed for the a priori scales 
in the present study. 
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5. Investigate the Use of Continuous Criterion for Prediction 

An alternative to a two group criterion should be Investigated • 

A multiple regression approach with a continuous criterion 

would seem to have the most promise. It would develop Inventory 

scales against a continuous criterion of satisfaction. An Individual 

could then be classified on the basis of a combination of 

manpower requirements In various fields and the satisfaction 

scores predicted for him In each of several fields. 

6. Examine Methodological Problems 

A number of methodological questions surround the traditional 
procedures used to key Interest Inventories. The effects of 
explicit selection on AQE scores are not clear; I.e., men are 
eligible for certain careers only within a given AQE score range. 
For example. It would not be necessary to use the Weather 
Observer's scale for anyone with an AQE General score of less 
than the minimum required score of 80. Thus, discrimination 
should be only between satisfied Weather Observers and those 
men-ln-general whose AQE score qualifies them. Such explicit 
selection could have considerable Impact, particularly when the 
occupational scale approach is used. 

Along the same line, there are undoubtedly other selection factors 
producing self-selection. To what extent these factors affect 
the Interest scales and decision rules arrived at Is not clear. 
It may be possible to perform range-restriction correlations of 
some kind of the wlthln-careei personnel, using a men-in-general 
sample to achieve more useful results. Both a theoretical and 
an empirical approach to these questions are needed. 
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7. Estimate Test-Retest Reliability 

The establishment of reasonable standards of measurement error 
for a psychological test Is paramount In a developmental effort. 
When an Instrument contains a significant degree of measurement 
error, Its usefulness as an Indicator of present status or a 
predictor of future behavior Is questionable. In psychological 
measurement, measurement error Is defined in terms of reliability 
coefficients and is frequently estimated by obtaining correlations 
between two administrations of a test. 

If a correlation between scores on two idministratlons is to 
serve as an estimate of reliability, the administrations must be 
conducted independently and under identical conditions. Further- 
more, the distributions of the test scores for the two administrations 
must be Identical. 

It Is recommended that such a reliability study be conducted under 
strict control. The subjects for this study should not be in 
basic training, but rather in the field. This would make the 
assumption of equal variance more likely to occur In practice. 
The test should be administered under supervised conditions on 
both occasions and under conditions in which the men are at ease 
so that measurement of their Interests can take place with 
minimum error. 

Establish Methodology for Differential Assignment 
No attempt was made to differentiate career interests among tvc 
or more of the various career fields at one time. The inventory 
may classify a man to an occupational area as opposed to men- 
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in-general. However, in the actual recruiting situation, the 
man who conceivably have interests identified to more than one 
career area must be assigned to only one area« In a situation such 
as this, the comparability of interest scores must be taken into 
consideration. To do this» an investigation should focus on 
interest scales which differentiate between a number of career 
fields being considered simultaneously. 
Develop Additional Scales 

The occupations represented in the present study were necessarily 
limited. It is strongly recommended that development of a greater 
variety of occupational scales than the eight in the present study. 
It is suggested, however, that this step not be carried out until 
some of the questions and issues addressed in the preceding 
recommendations have been answered « 
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DEPARTMENT OF THE AIR FORCE 

AFHRL PERSONNEL RESEARCH DIVISION (AFSC) 
LACKLAND AIR FORCE BASE. TEXAS 78236 



tmr TO 

ATTN OF: 



PEPP 




suuccT: Administration of USAF Vocational and Occupational Interest 
Survey (USAF SCN 7 3-3 3) 

TO: DPMQS/Survey Control Officer 

1, Reference is made to HQ USAF/ACMR letter, 20 Dec 1972, 
Vocational and Occupational Interest Choice Examination 
Survey. 

2, Educational Testing Service, under Government contract 
with the Personnel Research Division of the AF Human Resources 
Laboratory, is developing an occupational interest inventory 
(VOICE) for Air Force use. This inventory will eventually be 
used to assist in job placement of new recruits • In order to 
accomplish this research, these inventories need to be 
administered to a selected sample of Air Force personnel. 

3, Attached are inventory packets and a list of enlisted 
personnel assigned to units under your jurisdiction who were 
selected to participate in the field testing of this inventory. 
We are requesting your cooperation in distributing these 
packets. The listing includes identification data on all 
selected personnel and an indication of those personnel who 
are expected to arrive at a unit under your jurisdiction within 
the time period of the survey administration. If the airman 

is expected to arrive within the next two weeks after receipt 
of the survey packets, hold this material for him until he 
arrives. If personnel indicated on the roster have been 
transferred, we would appreciate your forwarding these materials 
to them as soon as possible. Respondents are requested to 
return all materials directly to Educational Testing Service 
upon completion. 

^. We greatly appreciate your assistance in the distribution 
of the enclosed packets. If you have any questions or require 
additional information, contact Dr. Guinn, Personnisl Research 
Division, Lackland AFB, Texas, AUTOVON 473-3967 • 

FOR THE COMMANDER 



2 Atch 

1. Roster 

2. Inventory Packets 





S. HOGGASn-4 Colonel, USAF 
, Personnel Research Division 
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DEPARTMENT OF THE AIR FORCE 

AFHRL PERSONNEL RESEARCH DIVISION (AFSC) 
LACKLAND AtR FORCE BASE, TEXA1> 78236 



REPLY TO 
ATTN Of I 



PEPP 



Follow-up on Administration of USAF Vocational and 
Occupational Survey (USAF SCN 7 3-33) 

DPMQS/Survey Control Officer 

1. Reference is made to HQ USAF/ACMR letter, 20 Dec 1972, 
Vocational and Occupational Choice Examination Survey* 

2, Approximately two weeks ago, you received a number of 
inventories to be distributed to personnel under your 
jurisdiction in support of a research project being conducted 
by Educational Testing Service (ETS) under contract with the 
Personnel Research Division. 



3. As of this 
as yet, return 
which were ori 
that as many o 
the inventory, 
the enclosed p 
attached roste 
encouraging th 
possible and r 



date, one or more 
ed the materials to 
ginally distributed 
f the selected pers 

we are again reque 
ackets to those per 
r. In addition, we 
em to complete the 
eturn it directly t 



of your personnel have not. 
Educational Testing Service 
Since it is very important 
onnei as possible complete 
sting that you distribute 
sonnel designated on the 

would appreciate your 
inventory as soon as 
o ETS. 



^. Thank you again for your cooperation and assistance in 
this project. If you have any questions, contact Dr. Guinn 
at the Personnel Research Division, Lackland AFB, Texas, 
AUTOVON 473-3967. 



FOR THE COMMANDER 



'O^JI 



ChieT, Person 




Colonel, USAF 
Research Division 



2 Atch 

1. Roster 

2 . Inventory Packets 
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REPLY TO 
ATTN OF: 



. PEPP 



AFHRL PERSONNEL RESEARCH DIVISION (AFSC) 
LACKLAND AIR FORCE BASE, TEXAS 78236 



DEPARTMENT OF THE AIR FORCE 




SUBJECT: Vocational and Occupational Interest Questionnaire (SCN 73-33) 



Selected USAF Enlisted Personnel 

1. The attached questionnaire, the Vocational and Occupational Interest 
Choice Examination (VOICE), is part of a study being conducted under con- 
tract by Educational Testing Service (ETS) for the Personnel Research 
Division of the Air Force Human Resources Laboratory. This questionnaire 
is designed to obtain information regarding your interests in various oc- 
cupational areas and job activities. The aim of the study is to help in 
assigning future Air Force recruits into career specialties corresponding 
more closely with their interests* In order to gain information to be 
used in this research, we are requesting a select group of enlisted per- 
sonnel to complete all five sections of VOICE. Since our project depends 
on your participation, it is very important that we receive your answers 
promptly. 

2. VOICE is not a test of ability; there are no right or wrong answers. 
Your best answers are those that accurately indicate your true feelings. 
The information requested is for official research purposes only and will 
NOT be placed in your personnel records or made available to your super- 
visor or commander. Read the directions carefully, then work quickly. 
Only your Social Security Account Number (SSAN) is required on the answer 
sheet which is placed inside the questionnaire packet. Do not write in 
the booklet; mark answers only on the answer sheet with the pencil pro- 
vided. Do not use a pen. The answer sheet will be scored by machine 

so make your marks dark and only within the boxes provided on your answer 
sheet. 

3. It is requested that the attached questionnaire be completed as soon 
as possible. Upon completion, return the questionnaire, answer sheet, and 
pencil in the self-addressed envelope directly to Educational Testing Ser- 
vice, Developmental Research Division, Princeton, New Jersey, 08540. 

4. Thank you for your cooperation in this project, 
FOR THE COMMANDER 




1 Atch 

Questionnaire packet 



Chief, Personnel Research Division 
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DEPARTMENT OF THE AIR FORCE 

AFHRL PERSONNEL RESEARCH DIVISION (AFSC) 
LACKLAND AIR FORCE FMSE. TEXAS 76236 



REPLY TO 
ATTN OF: 



PEPP 



SUBJECT: FoUow-tip OH CoiTiplotion o£ Vocational and Occupational Interest Questionnaire 

TO: Selected USAF Enlisted Personnel 

1. Approximately two weeks ago, a VOICE questionnaire was sent to you. 
As o£ today, your reply has not been received. Since it is very important 
that we obtain completed questionnaires from everyone contacted in this 
study so that the results o£ our research will be valid, we are enclosing 
another copy o£ the questionnaire to be completed and returned to Educa- 
tional Testing Service. I£ you have already mailed your previous question- 
naire, disregard this second request. 

2. VOICE is not a test of ability; there are no right or wrong answers. 
Your best answers are those that accurately indicate your true feelings. 
All of your answers wiU be kept CONFIDENTIAL and will be used for re- 
search purposes only. Read the directions carefully, then work quickly. 
When you have finished, return all materials provided in the self -addressed 
envelope to Educational Testing Service, Development.^ Riisearch Division, 
Princeton, New Jersey, 08540. 

3. Thank you again for your participation in this research project. 
FOR THE COMMANDER 




RALPR S. HOGGA'Hg/,vColonel, USAF , 
Chief, Personnel Research Division 



1 Atch 

Questionnaire packet 
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VOICE 

Vocational Occupational Interest Choice Examination 

Information regarding the VOICE questionnaire may be obtained by 
writing to the authors at Educational Testing Service, Princeton, 
New Jersey 08540. 
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APPENDIX B 



Maximum Likelihood Placement Using Eight Satisfied Career Groups 



ERIC 



-Ill- 
Maximum Likelihood Placement Using Eight Satisfied Career Groups 

.1*. 

An alternative placement strategy consisted of the following steps: 

1, Using satsified groups only , perform a discriminant analysis with 
the eight groups as dependent variables and the a priori scales 
as independent variables. 

2. Use the following function to make placement decisions 

^ij " ^ij ^j"^ ^ij ^ '^j ' ^ ' 

where X^^^^ ^ is a vector of discriminant scores for the ith individual 
taken as deviations from the j th group mean, D^^ is Lh? inverse of the 
j th group covariance matrix, and Pj is the a priori (or a posteriori) 
probability of membership in the jth group. 

In the present study, F was computed for each of the eight groups 
for each individual. The individual was then assigned to the group for 
which the function F was lowest in algebraic value, since the lowest 
algebraic value would correspond to the highest probability. The Pj 
were estimated in the half-samples as 

8 

N / E N , 
^ i-1 ^ 

where Nj is the number of satisfied personnel in group j in the half- 
sample. 

The discriminant functions and classification rules from one half- 
sample were applied to the other half-sample so that an estimate of the 
accuracy of the classification scheme could be made. 

ERIC 
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Tables 1 and 2 present the results of the eight-group discriminant 
analyses for the two half-samples using a priori scales. Since the 
percentage variance accounted for beyond the 4th latent root was minimal, 
only the first four discriminant functions were employed in the cross- 
validation. 

Table 3 shows the cross-validated predicted and actual placements 
using the first four discriminant functions and the function F derived 
from one half -sample to make placements in the other half-sample. The 
numbers in the diagonals represent hits and the off-diagonal entries 
represent misses. 

Though both hit rates are relatively low — 14.4 percent in Sample 1, 
and 19.1 percent in Sample 2 — both hit rates are higher than the expected 
hit rate given proportional random assignment and in Sample 2 the cross- 
validated hit rate exceeded the maximal blind strategy of assigning 
everyone to the largest group. 

Since the sample sizes were somewhat small for estimating the number 
of parameters involved, it may be of interest to examine the "shrinkage" 
which occurred as placement rules were first applied to the sample on 
which they were developed and then to the holdout sample. The hit rate for 
the rules developed on Sample 1 shrank from 20.5 percent (when applied to 
Sample 1) to 19.1 percent (when applied to Sample 2). The rules developed 
on Sample 2 showed a hit rate of 23.2 percent when applied to Sample 2 
which shrank to 14.4 percent when crossed to Sample 1. 

Though the results reported in Table 3 suggests that the method 
may not be overly promising, scoring procedures based on results for the 
total sample can be specified. If a computer is available the 13 a priori 
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scores (based on the final sets of items specified in the final report) 
can be entered as variables into the following set of equations: 

where X^^ in a vector of discriminant function deviation scores for 
individual i , expressed as deviations from the j th group mean, 
Bj is the jth group dispersion matrix and Pj is an a priori or 
assigned probability for the jth group. The indivi-mal would be assigned 
to the group for which F^_. was highest in algebraic value. 

The information necessary to perform these calculations is presented 
as Tables 4, 5, and 6 of the report. 

Table 4 shows the weights for the first four discriminant functions 
derived from data for the total sample. Table 5 presents the discriminant 
function means for each of the eight career field satisfied groups and Table 
presents the variance-covariance matrices for the eight groups. 

The vector X in the above equation can be rewritten as 



where B is a 4x13 matrix of discriminant function coefficients, is 

a 4x1 vector of means, and Y^^ is the 13x1 vector of a priori test scores 

for individual i 

If we let A = B" D?"^ M, ; a 13x1 vector 
j 3 j 

Cj = Dj-^ B ; a 13x13 matrix 

Kj = Mj Dj"^ Mj ; a scalar. 
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The classification equation can be 
scores 

Since 21og P , K and log D are 
can be expressed as 

where g = 21og P - K - log |d | . 



written in terms of the original 



2Aj] - - log |Dj| . 

all scalar quantities the equation 
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